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ABSTRACT 

This report is the second in a series based on data collected 
for an evaluation of the Class Size Reduction Program. This study examined 
the impact of the teacher on student achievement. The study used multilevel 
statistical techniques to examine the teaching strategies and techniques 
observed in the classroom that were significant predictors of student 
achievement as measured by the spring 2000 Stanford Achievement Test/9 
reading, mathematics, and language subtests. The analysis included 
controlling for student-level and teacher-level characteristics that might 
have otherwise biased the results. Some of the control variables at the 
student level included pretest national curve equivalent score (spring 1999), 
language classification, grade-level, and socioeconomic status (measured by 
free or reduced-price lunch) . The teacher-level predictors included 
credentials and years of teaching experience. Results indicate that there are 
specific observed teaching techniques that impact student achievement in 
reading and language. The use of classroom management skills is a significant 
predictor of reading achievement, and language achievement appears to be 
positively related to skills associated with individualization and engagement 
of students. The teaching behaviors measured did not predict mathematics 
achievement. Teaching status (permanent versus nonpermanent) had a positive 
impact on students' reading, mathematics, and language posttest scores 
(spring 2000). English language learners in grade 2 with more experienced 
teachers showed larger mathematics and language gains. A similar pattern was 
observed for third grade. Students in both second and third grades had been 
in reduced size classes for 3 years. Overall results suggest that teacher 
experience and status (credential) and certain classroom techniques improve 
student achievement. An appendix outlines the teaching strategies studied, 
and a second appendix describes the student and teacher samples. (Contains 9 
tables, 12 figures, and 19 references.) (SLD) 
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EXECUTIVE SUMMARY 



Introduction 

This report is the second in a series of three reports based on data collected during the 
1999-00 school year for an evaluation of the Class Size Reduction Program. The first report in 
this series examined the impact of class size reduction (CSR) on achievement among 3rd, 4th, 
and 5th grade students with different numbers of years of participation in the program. This 
report extends beyond the first by examining the role of the teacher in impacting student 
achievement. 

The purpose of this study was to use multilevel statistical techniques to examine which 
teaching strategies and techniques observed in the classroom were significant predictors of 
student achievement as measured by the spring 2000 SAT/9 reading, mathematics, and language 
subtests. The analysis included controlling for student-level and teacher-level characteristics that 
might have otherwise biased the results. Some of the control variables at the student-level 
included the following: pretest (spring 1999) NCE score, language classification, grade-level, 
and SES (free/reduced lunch). The teacher-level predictors included credentialing and years of 
teaching experience. 

Conventional wisdom suggests that effective teachers should increase the probability that a 
student will learn. Teachers possess an entire repertoire of teaching strategies, techniques, and 
characteristics that may or may not lead to student achievement. The focus of this analysis was to 
determine which strategies, techniques, and/or characteristics of the teachers resulted in 
increased student achievement on the SAT/9 reading, mathematics, and language subtests. 



Findings 



The multilevel analysis took into consideration the fact that students are “nested” or 
grouped within teachers. The results indicated that there were specific observed teaching 
techniques that impacted student achievement in reading and language. The use of classroom 
management skills was a significant predictor of reading achievement. Language achievement 
appeared to be positively related to those skills associated with individualization and engagement 
of students. However, the teaching behaviors we measured did not predict mathematics 
achievement. 

Additional findings from the multilevel analysis revealed that teaching status (permanent 
versus non-permanent) had a positive impact on students’ reading, mathematics, and language 
posttest scores (spring 2000 NCE scores). This impact appeared to be the strongest for reading 
and language, followed by mathematics. 

Exploratory analyses were then used to examine the relationships between teaching 
experience and teaching status by language classification. The analysis with regard to teaching 
experience indicated that English Language Learner (ELL) students in 2nd grade classrooms 
where the teacher had 3 to 10 years experience scored significantly larger adjusted mathematics 
and language gains than those students who had the least experienced teachers. The results for 
2nd grade English Only (EO) students suggest that those students who had the most experienced 
teachers scored significantly larger mathematics gains than those students who had the least 
experienced teachers. The effect size (ES) for this difference is educationally important (d = .20). 

The 2nd grade findings for teaching status indicate that those ELL students who had 
permanent teachers scored larger adjusted gains than those students with non-permanent teachers 
across all three SAT/9 tests. However, the only statistically significant difference between 
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permanent and non-permanent teachers was in reading. The findings for 2nd grade EO students 
indicated that students having permanent teachers, on the average, scored larger adjusted gains 
on all achievement tests than EO students with non-permanent teachers. The effect sizes for 
reading, language, and math were d = .40, d = .67, and d = .45, respectively. These effect sizes 
reflect a medium to large impact on adjusted gains due to teaching status. 

The findings for 3rd grade ELL students’ adjusted achievement gains were similar across 
categories of teaching experience. The analysis for 3rd grade EO students did not result in any 
significant differences between student achievement and categories of teaching experience. In 
retrospect, the trend for ELL students was very similar to the trend for EO students with respect 
to teaching experience. 

Third grade ELL students who had permanent teachers scored smaller adjusted losses on all 
achievement tests as compared to those students who had non-permanent teachers. The 3rd grade 
EO adjusted reading, mathematics, and language gains were larger for those students who had 
permanent teachers as compared to the adjusted gains for the students with non-permanent 
teachers. However, there were no statistically significant differences between students with 
permanent teachers as compared to those students with non-permanent teachers. 

Conclusions 

This study investigated the impact of teaching techniques and teacher characteristics in 
reduced size 2nd and 3rd grade classrooms using data from both classroom observations and 
matched student achievement scores. The analysis considered student characteristics and teacher 
characteristics. Both the 2nd and 3rd grade students had been in reduced size classes for three 
years. Thus, the average number of students per class meeting this criterion was less than 20. 
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While previous research suggests that smaller class size may help to improve student 
achievement, it is unclear how this outcome is related to the content of instruction in specific 
subject areas. Stasz and Stecher (2000) found that students in reduced size classes spent more 
time during language instruction writing narrative pieces. They also found that students engaged 
in mathematics instruction played mathematics games, and examined relationships using 
numbers. 

Stasz and Stecher (2000) examined other factors that may have affected their results. They 
compared teacher characteristics of those in reduced and non-reduced size mathematics classes. 
They found that there were a few significant differences in teacher attributes, such as having a 
master’s degree and staff development that may result in increased student achievement. 
However, Stasz and Stecher (2000) were unable to examine the relationship between 
instructional practices and student outcomes because they could not link the data to individual 
students. 

This study used multilevel modeling (HLM) to uncover the relationships between teaching 
strategies and characteristics, and student achievement. The results of the multilevel analysis 
revealed that after controlling for student-level variables such as language classification, grade 
level (2nd vs. 3rd), and spring 1999 SAT/9 NCE scores, the significant teacher-level predictors 
of SAT/9 spring 2000 NCE reading scores were teaching status (permanent vs. all others) and 
classroom management. The findings further indicated that teaching status was a significant 
predictor of mathematics and language outcomes. Individualized instruction was also a 
significant predictor of language outcomes. This means that on average, that teachers who were 
credentialed and experienced, had students who made the largest adjusted gains in reading, 
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mathematics, and language. Previous research shows the greater use of individualization in 
smaller size classes (Molnar, Smith, Zahonk, Palmer, Halbach, & Ehrle, 1999). 

The current study adds to the literature about the relationship between teaching techniques 
and behaviors and student achievement. The results presented in this paper have suggested that 
for 2nd and 3rd grade students in LAUSD, teacher experience and status (credential), as well as 
certain classroom techniques, improve student achievement. 
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The Relationship between Teacher Instructional Techniques and Characteristics 
and Student Achievement in Reduced Size Classes 

This report is the second in a series of three reports based on data collected during the 
1999-00 school year for an evaluation of the Class Size Reduction Program. The first report in 
this series examined the impact of class size reduction (CSR) on achievement among 3rd, 4th, 
and 5th grade students with different numbers of years of participation in the program. This 
report extends beyond the first by examining the role of the teacher in impacting student 
achievement. 

Conventional wisdom suggests that effective teachers should increase the probability that a 
student will leam. Teachers possess a repertoire of teaching strategies, techniques, and 
characteristics that may or may not lead to student achievement. The focus of this analysis was to 
determine which strategies, techniques and/or characteristics of the teachers resulted in increased 
student achievement on the SAT/9 reading, mathematics, and language subtests. 

Literature Review 

Research suggests that school factors, such as class size (Glass, Cahen, Smith, & Filby, 
1982; Mosteller, 1995) and teacher quality (Ferguson, 1991) influence student achievement. 
There has not been much empirical evidence linking teacher instructional techniques and student 
achievement. However, Darling-Hammond (2000) found that schools can make a difference in 
mathematics and a great portion of the difference is due to teacher preparation including 
credentialing. The Tennessee Value-Added Assessment System (Sanders & Rivers, 1996) 
illustrated large teacher-to-teacher differences in student learning. Sanders and Rivers also 
showed that teachers’ efforts were additive and cumulative. 
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Within the research on the role of the teacher in fostering student achievement, the focus 
has been on teacher qualifications (e.g. credentialing and years of experience). In response to this 
research, more than 25 states have enacted legislation to improve teacher qualifications (Darling- 
Hammond, 1997). These states have implemented improvements in teacher education, 
certification, professional development, and recruitment practices. 

Sanders and Rivers (1996) found that effective teachers are far more important to student 
learning than most other large reforms. Unfortunately, their research did not include an 
examination of explicit teaching techniques. 

Stasz and Stecher (2000) investigated the effects of smaller class sizes on teacher 
instructional techniques. Their results indicated that there were only a few differences found 
between reduced and non-reduced classes in teacher instructional practices. Stasz and Stecher 
(2000) were unable to examine the relationship between instructional practices and student 
outcomes directly. This was due to their inability to link teachers’ survey responses to student 
test scores. 

Research on teaching effectiveness has generally indicated that teachers with more teaching 
methods courses, more professional development, and more enthusiasm, have higher achieving 
students than teachers with lower levels on these indicators. Darling-Hammond (2000) found 
that measures of teacher preparation and credentialing were strongly related to student 
achievement in mathematics and reading, both before and after controlling for student poverty 
and language status. 

Measurement and Statistical Issues in Educational Data 
Residualized (Adjusted) NCE Gain Scores 

This study employed regression analysis to adjust the pretest scores for pre-existing group 
differences in order to estimate an adjusted or residualized gain score for each student. The 
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posttest scores were regressed on the pretest scores in order to obtain the residual scores. The 
residual scores represent improvements and/or decrements in student achievement. The adjusted 
difference scores will be referred to as “adjusted gains” in this report. 

Students “Nested” within Teachers 

Educational research usually involves nested or hierarchical data structures. This means that 
students are located, or “nested,” within teachers’ classrooms, and teachers are located, or 
“nested,” within schools. Traditional statistical techniques have not adequately considered this. 
Consequently, differences that may be due to unique effects teachers may have on student 
achievement are often not considered. Previous research has ignored the fact that students are 
located in different classrooms. The problems created by this approach were recognized (e.g. 
Burstein, 1980), but remained statistically intractable. There have been recent developments in 
statistical software that now enable researchers to examine these relationships (Bryk & 
Raudenbush, 1992). Multilevel modeling allows us to take into account the fact that students are 
located, or nested, within particular classrooms and to analyze effects that may be related to 
teachers. 

Purpose of Study 

The purpose of this study was to use multilevel statistical techniques to examine which 
teaching strategies and techniques observed in the classroom were significant predictors of 
student achievement as measured by the spring 2000 SAT/9 reading, mathematics, and language 
subtests. The analysis included controlling for student-level and teacher-level characteristics that 
might have otherwise biased the results. Some of the control variables at the student-level 
included the following: pretest (spring 1999) NCE score, language classification, grade-level, 
and SES (ffee/reduced lunch). The teacher-level predictors included credentialing and years of 
teaching experience. 
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Research Questions 



1) Were there any teacher-to-teacher differences in the outcome variables (SAT/9 reading, 
mathematics, and language NCE adjusted gain scores)? If so, what factors were related to 
these differences? 

2) Was there a relationship between student characteristics and achievement? 

Method 

Participants 

The participants were 44 randomly selected 2nd grade teachers and 47 randomly selected 
3rd grade teachers and their students from 50 elementary schools in LAUSD. There were 1835 
students in the sampled teachers’ classrooms. Only those students who took both the spring 1999 
and spring 2000 test administrations were used in this analysis (matched scores). The student 
data were obtained from the LAUSD Information Center Branch. The teacher data were either 
collected during the observations or obtained from files maintained by the LAUSD certificated 
personnel department. 

Design and Procedure 

Twelve trained observers visited classrooms during the 1999-00 school year. Observations 
took place during a 3-hour block of reading/language arts instruction on two occasions. 
Checklists were used to record teaching strategies and techniques that were exhibited in the 
classroom during the block of instruction. 

Teacher-Level Variables . The teacher instructional techniques and strategies are listed in 
Appendix A. The 20 instructional techniques and strategies were statistically reduced to three 
underlying factors that were used in the analysis as indicators of basic teaching techniques 
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observed in the classroom. 1 The three factors were as follows: 1) individualization and 
engagement; 2) redundancy, practice, modeling; and 3) classroom management. A fourth 
variable, learning time was calculated by taking the total minutes spent in learning activities 
during the observation period divided by the total number of minutes in the observation. Other 
teacher-level measures were as follows: 

• Number of years teaching at current school 

• Credential status 

Student-Level Variables . Student-level outcome measures were NCE reading, mathematics, 
and language SAT/9 posttest scores. Additional student-level characteristics that were considered 
in the analysis are listed below: 

• Language program coded as English language learner (ELL) versus all others 

• Free/reduced meal program participation (SES) 

• Grade (2nd or 3rd) 

Results 

Sample 

A description of the student and teacher sample is contained in Appendix B. The median 
number of years that teachers taught at their schools was 5 years and the number of years that 
teachers taught in the district was 8 years. Third grade teachers had fewer years teaching at their 
schools (Mdn=5) than 2nd grade teachers (Mdn=7). The majority of teachers were fully 
credential ed. 



1 Because the 20 instructional techniques were highly correlated, they were factor-analyzed (reduced) to three 
underlying factors or themes. These themes were used as proxy variables for instructional techniques. 
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Student Achievement 



The focus of this analysis was on the relationship between teaching behaviors and student 
achievement as measured by the SAT/9. There are different ways to measure the change in 
student achievement between two points in time. In the HLM analysis, pretest and posttest NCE 
scores were entered into the analysis. However, for all descriptive and inferential statistics 
reported in this study, adjusted/residualized gain scores were employed. In order to determine the 
unique contribution of teachers, it was necessary to first examine the relationships between 
student characteristics and adjusted gains. Did students’ adjusted gains in reading, mathematics, 
and language differ due to the student demographic characteristics? 

Table 1 



Adjusted NCE Gain Scores by Grade 







Reading 




Mathematics 




Language 




Grade 


n 


M 


SD 


n 


M SD 


n 


M 


SD 


2 


571 


2.15 


11.02 


602 


3.32 14.72 


582 


1.95 


15.40 


3 


662 


.10 


10.03 


713 


-1.10 13.03 


672 


.39 


12.55 



Table 1 indicates that 2nd grade students had significantly larger adjusted gains in reading 
and mathematics than 3rd grade students. 2 However, there were no statistically significant 
differences between 2nd and 3rd grade students’ language scores. It is interesting to note that 2nd 
grade students’ adjusted gains were larger across the three tests than those of 3rd grade students. 



2 Reading-F(3, 1252) = 8.56, £_= .00; Mathematics-F(3, 1334) = 12.35, 2 = .00. 



Table 2 



2nd Grade Adjusted Gains by Meal Program Participation 







Reading 




Mathematics 




Language 




Free/Reduced 

Lunch 


n 


M 


SD 


n 


M SD 


n 


M 


SD 


Yes 


443 


1.77 


11.26 


4.77 


2.65 14.91 


451 


.93 


15.71 


No 


102 


3.82 


9.88 


100 


6.19 12.86 


104 


6.65 


13.05 



Table 2 illustrates that the 2nd grade students who did not receive free/reduced lunch 
services scored larger adjusted gains than those students who did receive free/reduced lunch 
services. The adjusted reading gains were not significantly different. However, the adjusted math 
and language gains were significantly smaller for the lower SES students (free/reduced lunch 
program). 3 
Table 3 



3rd Grade Adjusted Gains by Meal Program Participation 







Reading 




Mathematics 




Language 




Free/Reduced 

Lunch 


n 


M 


SD 


n 


M SD 


n 


M 


SD 


Yes 


548 


-.55 


9.52 


589 


-1.65 12.74 


551 


-.33 


12.28 


No 


90 


4.26 


11.44 


90 


2.36 14.54 


89 


5.15 


14.12 



Table 3 indicates that 3rd grade students adjusted gains were significantly larger for those 
students not in the free/reduced lunch program as compared to those students who were in the 




3 Mathematics-t(575) = 2.44, j> = .02; Language-t(553) = 3.90, g_= .00. 
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free/reduced lunch program. 4 This means that the higher SES students had significantly larger 
adjusted gains than those of lower SES students. 

Table 4 



2nd Grade Adjusted Gains by Language Classification 



Language 

Classification 


Reading 




Mathematics 


Language 


n 


M 


SD 


n 


M 


SD 


n 


M 


SD 


ELL 


336 


1.64 


11.54 


3.64 


3.79 


15.05 


340 


.08 


15.49 


EO 


182 


2.51 


10.58 


185 


1.79 


15.08 


189 


3.13 


14.86 


IFEP 


42 


4.62 


9.58 


42 


4.96 


11.37 


61 


10.14 


14.58 


RFEP 


9 


2.42 


6.63 


9 


5.33 


15.21 


43 


6.91 


13.19 



The results presented in Table 4 indicate that among 2nd grade students there were no 
statistically significant differences in adjusted gains due to language classification for ELL and 
EO students. In reading and language, the EO students outperformed the ELL students. 
However, in mathematics, the reverse was true. The sample sizes for the two other language 
classifications are too small to make any inferences. 



4 Reading-t(659) = 3.78, jd = .00; Mathematics-t(677) = 2.48, jd = .02; Language- 1(638) = 3.47, g = .00. 
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3rd Grade Adjusted Gains by Language Classification 



Language 

Classification 


Reading 




Mathematics 


Language 


n 


M 


SD 


n 


M 


SD 


n 


M 


SD 


ELL 


367 


-.70 


9.67 


407 


-1.83 


13.18 


374 


-.10 


12.39 


EO 


187 


1.43 


11.10 


195 


-2.27 


12.82 


192 


-.38 


13.09 


IFEP 


63 


.86 


9.65 


64 


3.46 


11.98 


61 


2.15 


12.93 


RFEP 


44 


.15 


8.43 


43 


3.71 


11.68 


43 


4.15 


10.54 



Table 5 demonstrates that in reading, EO students outperformed ELL students. However, in 
mathematics and language, ELL students had smaller adjusted losses than EO students. The 
differences in adjusted gains were not statistically different due to language classification for 3rd 
grade students. 



Table 6 

2nd Grade Adjusted Gains by Ethnicity 



Ethnicity 


Reading 




Mathematics 




Language 


n 


M 


SD 


n 


M 


SD 


n 


M 


SD 


Hispanic/Latino 


397 


2.07 


11.35 


433 


3.49 


14.60 


404 


.79 


15.51 


Black 


67 


-.10 


11.35 


64 


-5.14 


15.82 


69 


-2.08 


16.12 


White 


53 


4.90 


10.19 


56 


7.94 


11.42 


56 


7.59 


13.22 


Asian 


26 


4.64 


7.99 


23 


9.39 


14.87 


26 


7.37 


12.09 



In this sample, among 2nd grade students, Black students had the largest adjusted losses. 
White students and Asian students had the largest adjusted reading and language gains. 
Additionally, Asian students had the largest adjusted mathematics gain scores. There were no 



statistically significant results for reading. However, the findings suggested that there were 
statistically significant differences in adjusted gains for mathematics and language. 5 

Specifically, for mathematics, there was a significant difference in adjusted gains between 
Black students and all other ethnic groups tabled above. In language, White students and Asian 
students scored significantly larger adjusted gains than Black students. 

Table 7 



3rd Grade Adjusted Gains by Ethnicity 



Ethnicity 


Reading 




Mathematics 




Language 


n 


M 


SD 


n 


M 


SD 


n 


M 


SD 


Hispanic/Latino 


497 


-.22 


9.53 


544 


-1.56 


12.83 


508 


-.31 


11.99 


Black 


57 


1.23 


9,19 


61 


-4.77 


11.16 


59 


-2.66 


11.86 


White 


53 


4.20 


13.26 


53 


1.12 


14.67 


52 


1.49 


14.89 


Asian 


36 


.54 


12.70 


36 


6.37 


13.37 


35 


9.31 


12.87 



Among 3rd grade students, Asian students had the largest adjusted mathematics and 
language gains, whereas White students had the largest adjusted reading gains. Black students 
had the largest adjusted losses in mathematics and language. Hispanic/Latino students had the 
smallest adjusted gains and losses across all three tests. There were statistically significant 
differences between ethnic groups for mathematics and language. 6 In mathematics, there was a 
significant difference between Asian students’ and Black students’ adjusted gains, and Asian 
students and Hispanic students’ adjusted gains. 

In summary, for 2nd and 3rd grade students, adjusted gains were largest for Asian students 
in mathematics and language. White students had the largest adjusted gains in reading and Black 



s Mathematics-F(5,593) = 10.54, jj = .00; Language-F(5,573) = 5.31, £ = .00. 

6 Mathematics-F(6,706) = 6.45, j> = .00; Language-F(6,665) = 7.97, g = .00 
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students had the largest adjusted losses in mathematics and language. The magnitude of the 
adjusted gains was different for 2nd grade students as compared with those of 3rd grade students. 

In order to account for the hierarchical or nested structure of the data (students located in 
teachers’ classrooms and teachers located in schools), a multilevel analysis was employed. 
Hierarchical linear modeling (HLM; Bryk, & Raudenbush, 1992) was used to examine the 
relationship between teaching strategies and student achievement. 



Results of Multilevel Analysis 
Table 8 

Classroom-to-Classroom Differences in Student Outcomes 





Classroom-to Classroom 


SAT/9 Subtests 


Differences 


Reading 


18% 


Mathematics 


34% 


Language 


25% 



Table 8 shows the percent difference in the outcome variables by classroom. This table 
indicates that there was a considerable difference in student achievement across teachers 
(classrooms). For example, more than one-third of the variance in mathematics NCE scores was 
between classrooms. What factors may have influenced this difference between 
classrooms/teachers? 

One of the purposes of the multilevel analysis was to examine the relationship between 
teaching status and student achievement. Our preliminary results indicated that there was an 
interaction between teaching status and years of teaching experience. The findings revealed that 
credentialed teachers with less than 3 years of teaching experience had students with 
significantly smaller gains (larger losses) than those students who had credentialed teachers with 



